Segmenting Hand-Drawn Strokes

Pen-based interfaces utilize sketch recognition so users can create and interact with complex, graphical systems via drawn input. In order for people to freely draw within these systems, users' drawing styles should not be constrained. The low-level techniques involved with sketch recognition must then be perfected, because poor low-level accuracy can impair a user's interaction experience. Corner finding, also known as stroke segmentation, is one of the first steps to free-form sketch recognition. Corner finding breaks a drawn stroke into a set of primitive symbols such as lines, arcs, and circles, so that the original stoke data can be transformed into a more machine-friendly format. By working with sketched primitives, drawn objects can then be described in a visual language, noting what primitive shapes have been drawn and the shapes? geometric relationships to each other. We present three new corner finding techniques that improve segmentation accuracy. Our first technique, MergeCF, is a multi-primitive segmenter that splits drawn strokes into primitive lines and arcs. MergeCF eliminates extraneous primitives by merging them with their neighboring segments. Our second technique, ShortStraw, works with polyline-only data. Polyline segments are important since many domains use simple polyline symbols formed with squares, triangles, and arrows. Our ShortStraw algorithm is simple to implement, yet more powerful than previous polyline work in the corner finding literature. Lastly, we demonstrate how a combination technique can be used to pull the best corner finding results from multiple segmentation algorithms. This combination segmenter utilizes the best corners found from other segmentation techniques, eliminating many false negatives (missed primitive segmentations) from the final, low-level results. We will present the implementation and results from our new segmentation techniques, showing how they perform better than related work in the corner finding field. We will also discuss limitations of each technique, how we have sought to overcome those limitations, and where we believe the sketch recognition subfield of corner finding is headed

Blues for Gary: Design Abstractions for a Jazz Improvisation Assistant

AbstractWe describe the design and implementation of a tool to help students learn the art of jazz improvisation. The tool integrates elements of database, AI in the form of automatic melody generation, and human interface design. We describe the philosophy of using several coordinated mini-languages to provide user specifications for various aspects of the tool, including melody and chord representation, styles, melody generation, and other musical knowledge

EUROGRAPHICS Workshop on Sketch-Based Interfaces and Modeling (2007) M. van de Panne, E. Saund (Editors) Designing a Sketch Recognition Front-End: User Perception of Interface Elements

Programs that can recognize students ’ hand-drawn diagrams have the potential to revolutionize education by breaking down the barriers between diagram creation and simulation. Much recent work focuses on building robust recognition engines, but understanding how to support this new interaction paradigm from a user’s perspective is an equally important and less well understood problem. We present a user study that investigates four critical sketch recognition user interface issues: how users integrate the process of triggering recognition into their work, when users prefer to indicate which portions of the diagram should be recognized, how users prefer to receive recognition feedback, and how users perceive recognition errors. We find that user preferences emphasize the importance of system reliability, the minimization of distractions, and the maximization of predictability

Blues for Gary: Design Abstractions for a Jazz Improvisation Assistant

We describe the design and implementation of a tool to help students learn the art of jazz improvisation. The tool integrates elements of database, AI in the form of automatic melody generation, and human interface design. We describe the philosophy of using several coordinated mini-languages to provide user specifications for various aspects of the tool, including melody and chord representation, styles, melody generation, and other musical knowledge